feat(batch): Support log iter batch for table or mv #16487
Conversation
github-actions
bot
added
Invalid PR Title
type/feature
ci/run-e2e-single-node-tests
labels
xxhZs
changed the title
xxhZs
force-pushed
the
xxh/support-log-iter
branch
from
0590804 to
da9d45d
Compare
xxhZs
force-pushed
the
xxh/support-log-iter
branch
from
ab1f805 to
24a6a38
Compare
xxhZs
force-pushed
the
xxh/support-log-iter
branch
from
080b2ec to
ef42385
Compare
xxhZs
force-pushed
the
xxh/support-log-iter
branch
from
ef42385 to
11fd241
Compare
| } | ||
| Err(e) => Err(e), | ||
| }); | ||
| let chunk = collect_data_chunk(&mut iter, &schema, Some(chunk_size)) |
There was a problem hiding this comment.
I think we need to do it in a loop. Currently if there is more than chunk_size rows in iter, we will only collect the first chunk_size rows.
There was a problem hiding this comment.
Are you referring to the loop above, or is it a nested loop
There was a problem hiding this comment.
Oops. I thought the iter is created inside the loop. So now we won't miss any rows.
nits: we can do iter.map(|r| { ... }) before the pin_mut, so that we don't have to recreate the map stream in every iteration.
| } | ||
| Err(e) => Err(e), | ||
| }); | ||
| let chunk = collect_data_chunk(&mut iter, &schema, Some(chunk_size)) |
There was a problem hiding this comment.
Oops. I thought the iter is created inside the loop. So now we won't miss any rows.
nits: we can do iter.map(|r| { ... }) before the pin_mut, so that we don't have to recreate the map stream in every iteration.
xxhZs
force-pushed
the
xxh/support-log-iter
branch
from
f8fe040 to
e430e7f
Compare
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
We support batch calling the iter_log interface to query the log of a certain mv or table
Checklist
./risedev check(or alias,./risedev c)Documentation
Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.